PATENT 



WHAT IS CLAIMED IS: 

1. For use in a video display system capable of displaying a 
video program, a system for creating a multimedia summary of said 
video program, said system comprising: 

a multimedia summary generator capable of obtaining a 
transcript of the text of said video program and capable of 
obtaining audio-video segments of said video program, 

wherein said multimedia summary generator is capable of 
combining portions of said transcript and portions of said audio- 
video segments to create a multimedia summary of said video 
program. 

2. The system as claimed in Claim 1 wherein said multimedia 
summary generator is capable of creating said multimedia summary by 
selecting an audio-video segment that relates to a topic of said 
video program, and by adding said topic and said audio-video 
segment to said multimedia summary. 
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3. The system as claimed in Claim 2 wherein said multimedia 
summary generator comprises: 

a controller capable of executing computer software 
instructions contained within a memory coupled to said controller 
to create said multimedia summary of said video program by 
identifying at least one topic cue in said transcript of said video 
program, and by selecting at least one audio-visual template 
associated with said at least one topic cue, and by adding said 
topic cue and said audio-visual template to said multimedia 
summary . 

4 . The system as claimed in Claim 3 wherein said controller 
is capable of executing computer software instructions contained 
within a memory coupled to said controller to create said 
multimedia summary of said video program by identifying at least 
one subtopic cue for said at least one topic of said video 
program, and by selecting at least one audio-visual template 
associated with said at least one subtopic cue, and by adding said 
subtopic cue and said audio-visual template to said multimedia 
summary . 
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1 5 . The system as claimed in Claim 3 wherein said controller 

2 is capable of executing: 

3 a domain identification application capable of identifying a 

4 type of said video program; 

5 a topic cue identification application capable of identifying 

6 at least one topic cue in said transcript of said video program; 

7 a subtopic cue identification application capable of 

8 identifying at least one subtopic cue in said at least one topic 

9 of said video program; and 

10 an audio-visual template identification application capable of 

n identifying at least one audio-visual template associated with said 

12 at least one topic cue, and capable of identifying at least one 

13 audio-visual template associated with said at least one subtopic 

14 cue . 

1 6. The system as claimed in Claim 4 wherein said controller 

2 is capable of executing computer software instructions contained 

3 within a memory coupled to said controller to create an entry point 

4 for each topic that will allow a viewer to access each topic in 

5 said multimedia summary, and to create an entry point for each 

6 subtopic that will allow a viewer to access each subtopic in said 

7 multimedia summary, 
l 
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7. A video display system capable of creating a multimedia 
summary of a video program, said video display system comprising: 

a multimedia summary generator capable of obtaining a 
transcript of the text of said video program and capable of 
obtaining audio-video segments of said video program, 

wherein said multimedia summary generator is capable of 
combining portions of said transcript and portions of said audio- 
video segments to create a multimedia summary of said video 
program. 

8. The video display system as claimed in Claim 7 wherein 
said multimedia summary generator is capable of creating said 
multimedia summary by selecting an audio-video segment that relates 
to a topic of said video program, and by adding said topic and said 
audio- video segment to said multimedia summary. 
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1 9. The video display system as claimed in Claim 8 wherein 

2 said multimedia summary generator comprises: 

3 a controller capable of executing computer software 

4 instructions contained within a memory coupled to said controller 

5 to create said multimedia summary of said video program by 

6 identifying at least one topic cue in said transcript of said video 

7 program, and by selecting at least one audio- visual template 

8 associated with said at least one topic cue, and by adding said 

9 topic cue and said audio-visual template to said multimedia 
10 summary. 

1 10. The video display system as claimed in Claim 9 wherein 

2 said controller is capable of executing computer software 

3 instructions contained within a memory coupled to said controller 

4 to create said multimedia summary of said video program by 

5 identifying at least one subtopic cue for said at least one topic 

6 of said video program, and by selecting at least one audio-visual 

7 template associated with said at least one subtopic cue, and by 

8 adding said subtopic cue and said audio-visual template to said 

9 multimedia summary. 

l 
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11. The video display system as claimed in Claim 9 wherein 
said controller is capable of executing: 

a domain identification application capable of identifying a 
type of said video program; 

a topic cue identification application capable of identifying 
at least one topic cue in said transcript of said video program; 

a subtopic cue identification application capable of 
identifying at least one subtopic cue in said at least one topic 
of said video program; and 

an audio-visual template identification application capable of 
identifying at least one audio-visual template associated with said 
at least one topic cue, and capable of identifying at least one 
audio-visual template associated with said at least one subtopic 
cue . 
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12. The video display system as claimed in Claim 10 wherein 
said controller is capable of executing computer software 
instructions contained within a memory coupled to said controller 
to create an entry point for each topic that will allow a viewer to 
access each topic in said multimedia summary, and to create an 
entry point for each subtopic that will allow a viewer to access 
each subtopic in said multimedia summary. 

13 . For use in a video display system capable of displaying a 
video program, a method for creating a multimedia summary of said 
video program, said method comprising the steps of: 

obtaining a transcript of the text of said video program in a 
multimedia summary generator; 

obtaining audio-video segments of said video program in said 
multimedia summary generator; and 

combining portions of said transcript and portions of said 
audio-video segments in said multimedia summary generator to create 
said multimedia summary of said video program. 
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1 14 . The method as claimed in Claim 13 wherein the step of 

2 combining portions of said transcript and portions of said audio- 

3 video segments in said multimedia summary generator to create said 

4 multimedia summary of said video program comprises: 

5 selecting an audio-video segment that relates to a topic of 

6 said video program; and 

7 adding said topic and said audio-video segment to said 

8 multimedia summary. 

l 
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1 15. The method as claimed in Claim 14 further comprising the 

2 steps of : 

3 receiving in a multimedia summary generator instructions from 

4 computer software stored in a memory coupled to said multimedia 

5 summary generator; 

6 executing said instructions in said multimedia summary 

7 generator to identify at least one topic cue in said transcript of 

8 said video program; 

9 executing said instructions in said multimedia summary 

10 generator to select at least one audio-visual template associated 

11 with said at least one topic cue; and 

12 executing said instructions in said multimedia summary 

13 generator to add said topic cue and said audio-visual template to 

14 said multimedia summary. 

l 
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1 16. The method as claimed in Claim 15 further comprising the 

2 steps of : 

3 receiving in a multimedia summary generator instructions from 

4 computer software stored in a memory coupled to said multimedia 

5 summary generator; 

6 executing said instructions in said multimedia summary 

7 generator to identify at least one subtopic cue for said at least 

8 one topic of said video program; 

9 executing said instructions in said multimedia summary 

10 generator to select at least one audio-visual template associated 

11 with said at least one subtopic cue; and 

12 executing said instructions in said multimedia summary 

13 generator to add said subtopic cue and said audio-visual template 

14 to said multimedia summary. 

l 
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1 17. The method as claimed in Claim 15 further comprising the 

2 steps of: 

3 identifying a type of said video program with a domain 

4 identification application; 

5 identifying at least one topic cue in said transcript of said 

6 video program with a topic cue identification application; 

7 identifying at least one subtopic cue in said at least one 

8 topic of said video program with a subtopic cue identification 

9 application; 

10 identifying at least one audio-visual template associated with 

n said at least one topic cue with an audio-visual template 

12 identification application; and 

13 identifying at least one audio-visual template associated with 

14 said at least one subtopic cue with said audio-visual template 

15 identification application. 

l 
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18. The method as claimed in Claim 16 further comprising the 
steps of : 

receiving in a multimedia summary generator instructions from 
computer software stored in a memory coupled to said multimedia 
summary generator; 

executing said instructions in said multimedia summary 
generator to create an entry point for each topic that will allow a 
viewer to access each topic in said multimedia summary; and 

executing said instructions in said multimedia summary 
generator to create an entry point for each subtopic that will 
allow a viewer to access each subtopic in said multimedia summary. 
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19. For use in a video display system capable of displaying a 
video program, computer-executable instructions stored on a 
computer-readable storage medium for creating a multimedia summary 
of said video program, the computer-executable instructions 
comprising the steps of: 

obtaining a transcript of the text of said video program in a 
multimedia summary generator; 

obtaining audio-video segments of said video program in said 
multimedia summary generator; and 

combining portions of said transcript and portions of said 
audio-video segments in said multimedia summary generator to create 
said multimedia summary of said video program. 

20. The computer-executable instructions stored on a 
computer-readable storage medium as claimed in Claim 19 wherein the 
step of combining portions of said transcript and portions of said 
audio-video segments in said multimedia summary generator to create 
said multimedia summary of said video program comprises: 

selecting an audio-video segment that relates to a topic of 
said video program; and 

adding said topic and said audio-video segment to said 
mul t imedi a summary . 
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1 21. The computer-executable instructions stored on a 

2 computer-readable storage medium as claimed in Claim 2 0 further 

3 comprising the steps of: 

4 receiving in a multimedia summary generator instructions from 

5 computer software stored in a memory coupled to said multimedia 

6 summary generator; 

7 executing said instructions in said multimedia summary 

8 generator to identify at least one topic cue in said transcript of 

9 said video program; 

10 executing said instructions in said multimedia summary 

n generator to select at least one audio-visual template associated 

12 with said at least one topic cue; and 

13 executing said instructions in said multimedia summary 

14 generator to add said topic cue and said audio-visual template to 
is said multimedia summary. 

i 
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1 22. The computer-executable instructions stored on a 

2 computer-readable storage medium as claimed in Claim 21 further 

3 comprising the steps of: 

4 receiving in a multimedia summary generator instructions from 

5 computer software stored in a memory coupled to said multimedia 

6 summary generator; 

7 executing said instructions in said multimedia summary 

8 generator to identify at least one subtopic cue for said at least 

9 one topic of said video program; 

io executing said instructions in said multimedia summary 

n generator to select at least one audio-visual template associated 

12 with said at least one subtopic cue; and 

13 executing said instructions in said multimedia summary 

14 generator to add said subtopic cue and said audio-visual template 

15 to said multimedia summary. 



l 
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23. The computer-executable instructions stored on a 

computer-readable storage medium as claimed in Claim 21 further 

comprising the steps of: 

identifying a type of said video program with a domain 

identification application; 

identifying at least one topic cue in said transcript of said 

video program with a topic cue identification application; 

identifying at least one subtopic cue in said at least one 

topic of said video program with a subtopic cue identification 

applications- 
identifying at least one audio-visual template associated with 

said at least one topic cue with an audio-visual template 

identification application; and 

identifying at least one audio-visual template associated with 

said at least one subtopic cue with said audio-visual template 

identification application. 
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1 24. The computer-executable instructions stored on a 

2 computer- readable storage medium as claimed in Claim 22 further 

3 comprising the steps of: 

4 receiving in a multimedia summary generator instructions from 

5 computer software stored in a memory coupled to said multimedia 

6 summary generator; 

7 executing said instructions in said multimedia summary 

8 generator to create an entry point for each topic that will allow a 

9 viewer to access each topic in said multimedia summary; and 

10 executing said instructions in said multimedia summary 

11 generator to create an entry point for each subtopic that will 

12 allow a viewer to access each subtopic in said multimedia summary. 

1 25. For use in a video display system capable of displaying a 

2 video program, a multimedia summary of a video program comprising 

3 at least one audio-visual segment of said video program. 

1 26. The multimedia summary of a video program as claimed in 

2 Claim 25 further comprising at least one portion of a transcript of 

3 said video program. 



l 
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1 27. The multimedia summary of a video program as claimed in 

2 Claim 25 comprising at least one audio-visual segment of said video 

3 program that relates to at least one topic of said video program. 

1 28. The multimedia summary of a video program as claimed in 

2 Claim 2 7 comprising at least one audio-visual segment of said video 

3 program that relates to at least one subtopic in said at least one 

4 topic of said video program. 

1 29. The multimedia summary of a video program as claimed in 

2 Claim 25 wherein said multimedia summary is capable of displaying 

3 one of : 

4 text from said video program, audio from said video program, a 

5 single video frame from said video program, a video segment 

6 comprising a series of video frames from said video program, and 

7 an audio-visual segment comprising audio from said video program 

8 and a series of video frames from said video program. 

l 
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1 30. The multimedia summary of a video program as claimed in 

2 Claim 27 comprising a plurality of audio-visual segments of said 

3 video program, wherein each of said plurality of audio-visual 

4 segments relates to a topic of said video program. 

1 31. The multimedia summary of a video program as claimed in 

2 Claim 3 0 further comprising a topic entry point associated with 

3 each of said plurality of audio-visual segments that relates to a 

4 topic, in which each topic entry point allows a viewer to access 

5 the audio-visual segment associated with said topic. 

1 32. The multimedia summary of a video program as claimed in 

2 Claim 3 0 comprising a plurality of audio-visual segments of said 

3 video program, wherein each of said plurality of audio-visual 

4 segments relates to a subtopic of a topic of said video program. 

1 33. The multimedia summary of a video program as claimed in 

2 Claim 32 further comprising a subtopic entry point associated with 

3 each of said plurality of audio-visual segments that relates to a 

4 subtopic, in which each subtopic entry point allows a viewer to 

5 access the audio-visual segment associated with said subtopic. 

l 
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1 34. For use in a video display system capable of displaying a 

2 video program, a multimedia summary of a video program comprising 

3 a plurality of audio-visual segments of said video program 

4 that relate to at least one topic of said video program; and 

5 at least one topic entry point associated with said plurality 

6 of audio-visual segments that relate to said at least one topic of 

7 said video program, in which said at least one topic entry point 

8 allows a viewer to access the plurality of audio-visual segments 

9 associated with said topic. 

1 35. The multimedia summary of a video program as claimed in 

2 Claim 34 further comprising 

3 a plurality of audio-visual segments of said video program 

4 that relate to at least one subtopic of said at least one topic of 

5 said video program; and 

6 at least one subtopic entry point associated with said 

7 plurality of audio-visual segments that relate to said at least one 

8 subtopic of said at least one topic of said video program, in which 

9 said at least one subtopic entry point allows a viewer to access 
10 the plurality of audio-visual segments associated with said 
n subtopic. 

l 
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1 36. The method as claimed in Claim 13, said method further 

2 comprising the steps of: 

3 obtaining an image of a face of a person in said video program 

4 with audio-visual template identification application after said 

5 person first appears in said video program; 

6 subsequently confirming the identity of said person by 

7 checking at least one identifying characteristic of said person; 

8 and 

9 adding said image of said person to said multimedia summary 
10 after the identity of said person has been confirmed. 

1 37. The method as claimed in Claim 36 wherein said at least 

2 one identifying characteristic of said person comprises one of: 

3 an identification of the face of said person, an 

4 identification of the voice of said person, and a name plate of 

5 said person. 

1 38. The method as claimed in Claim 36 wherein at least one 

2 identifying characteristic of said person comprises an 

3 identification of the face of said person and an identification of 

4 the voice of said person. 
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